3D Doodling Wand
Outline An affordable and simple-to-use wand that will allow anybody to create 3D drawings on their computer. Each wand will have a cube on the end with a patterned marker on each face. A normal webcam will capture the markers and use them to calculate the 3D location/orientation of the wand. When a button is pressed the wand will go into ‘drawing mode’, where any movement of the wand will produce a vector that can be used to build up a 3D virtual object. The wand will also be integrated into existing applications (e.g. Games or design software) to provide a cheap & effective 6DOF input device. The standard wand cube will be free to print out for anyone, using keyboard buttons for the drawing commands. A special bluetooth wand with command buttons can also be built based around an ATtiny85 chip for <£10. Collaborators Gregory Hough (Project Lead) Sophie Bullock Samantha & Tristan Aplin Contact details The project lead can be contacted at Gregory.Hough2@mail.bcu.ac.uk Objectives * To develop a doodling environment in OpenGL with all the necessary tools (which need to be identified heuristically). * Development of an application that can accurately track the location and orientation of the markers with no prior calibration required by the user using OpenCV. * Develop a “smart-wand” with Bluetooth capabilities for easy drawing. * Integrate it with existing applications (e.g. Blender, Unity, Mudbox) Development The current state of development is pre-alpha. A prototype has been developed in Python that uses the FLANN bindings in OpenCV to track the individual faces of the doodling wand (currently limited to one). A series of algorithms are used to interpret the movement and rotation of the wand as 3D cursor movements. One current issue is the low rotation angle (<30'°') that is allowed using the current implementation. We propose two solutions - 1. That a dodecahedron could be used instead of a cube due to the lower angle between the faces and 2. That a more robust approach that can handle rotation angles of >45'''° '''be explored (perhaps SURF). These are currently being explored. Another current feature is that the user is able to draw their own designs for each face using a marker pen and select it from the camera image when the Doodling Wand is opened. We feel as thought this feature would make it more attractive for use with children playing video games and allow the creation of more aesthetically appealing wands for commercial use. As of yet, the smart wand has not been developed. The equipment has been bought. Equipment The tracking system is designed to work on a standard Windows PC to ensure wide use. The SmartWand will be developed using a microcontroller (perhaps an ATmega328 or ATiny85), a bluetooth module and the appropriate buttons. Support needs Support is required to help: * Develop the tracking module using the OpenCV libraries in Python. * Develop a “smart-wand” with Bluetooth capabilities for easy drawing. * Integrate it with existing applications. * Heuristic analysis with existing applications to determine how best to implement the wand. * User-testing. * Development of an OpenGL based doodling environment. Milestones # Develop an alpha prototype of the wand using existing feature detection methods. Transform the location and rotation of the tracked wand faces in the image and interpret it as motion for a 3D cursor. # Implement the wand into an existing application to test use. Proposed to be Blender or Unity. Concurrently develop OpenGL doodling environment. # Perform heuristic analysis in the application to guide next stage of development. # Develop wand to beta prototype according to the findings of the heuristic analysis - including smart wand functionality. # Distribute to users for testing. # Refine system based on user testing suggestions.